Display device and display method

ABSTRACT

The present invention includes a display device for displaying an image and receiving light simultaneously or alternately. The display device includes: a plurality of display elements for displaying the image on a display surface of the display device by emitting light; a plurality of light receiving elements for receiving light incident on the display surface formed by the display elements; a storing unit for storing, as an initial value, a difference between two amounts of light received by the light receiving elements; and a detecting unit for detecting a state of contact with or proximity to the display surface and subtracting the initial value stored by the storing unit from a difference between the two detected amounts of received light.

CROSS REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese PatentApplication JP 2005-203259 filed in the Japanese Patent Office on Jul.12, 2005, the entire contents of which being incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a display device and a display methodsuitable for application to a liquid crystal display and an organic EL(electroluminescence) display, for example, and particularly to displaytechniques that enable light reception in parallel with light emission.

2. Description of the Related Art

In related art, when a touch panel for allowing an operation by touchinga display screen of a display device such as a television receiver orthe like is formed on the display screen, the tough panel separate fromthe display device is put over the display screen.

As a constitution using a separate touch panel, there is for example aconstitution having a thin transparent input detecting device affixed ona screen. This is a touch sensor using a conductive film. There are forexample pressure-sensitive type devices detecting pressure, capacitancetype devices changed by contact with a human body, and the like. Inaddition, there are devices referred to as an electromagnetic inductiontype, which allows a position to be input using a special pen. Thesedevices have a structure in which the surface of a display panel hasanother special panel for position detection placed thereon.

While touch detection principles for these devices using the detectionpanel placed on the display panel are easy, a decrease in displayquality inevitably occurs because some member is placed on the displaypanel. In addition, since a detection method is mainly a method ofdetecting a capacitance change, it is difficult to detect inputs at twoor more positions simultaneously.

Touch panel systems in which no panel is added on the surface include anoptical type. A combination of a light emitting element (light emittingdiode or the like) and a phototransistor is arranged on an upper, alower, a left and a right side of a panel. A position is detected on thebasis of light blocked by a finger or the like. Such an optical typedoes not cause a decrease in display quality, but is not suitable forportable devices because of the large scale of the devices locatedaround the periphery of the display device.

In order to eliminate the inconveniences of these touch panels inrelated art, devices have recently been proposed in which the screen ofthe display devices functions as a touch panel as it is without aseparate touch panel being provided. Japanese Patent Laid-Open No.2004-127272 (FIG. 5) discloses such a display device that performs lightemission and light reception in parallel.

An example of such a display device that performs light emission andlight reception in parallel is configured such that for example display(light emission) of light emitting elements for image display, whichelements are arranged on a display surface, is made intermittently, acharge corresponding to received light is accumulated in the lightemitting elements themselves in periods when the light emission isstopped, and the accumulated charge is read. As a display device thatmakes such a configuration possible, there is for example an organic ELdisplay. In addition, it is proposed that when display elementsthemselves do not have a function of receiving light (accumulatingcharge) as in the case of liquid crystal displays, light receivingelements are arranged adjacent to the display elements and light isreceived by the light receiving elements in periods when display (lightemission) is stopped.

SUMMARY OF THE INVENTION

When this type of display device that emits light and receives lightsimultaneously is formed, the state of a received light signal is variedgreatly at times of light reception due to effects of extraneous light.Specifically, supposing that touching of a display screen with a fingeror the like is to be detected, for example, light receiving conditionsdiffer greatly between a state in which received light is detectedwithin a dark room and a state in which received light is detectedoutside in bright daylight. It is therefore difficult to detect contactwith the surface of the display device or the like under uniform lightreceiving conditions.

Hence, when the display device is applied to a device usable bothoutside and inside, as in a case where the display device is applied tothe display panel of a portable electronic device, for example, somemeasure to deal with changes in intensity of extraneous light isrequired. However, it is difficult to take such a measure easily in analready proposed display device that emits light and receives lightsimultaneously.

In addition, noise is caused by an internal configuration of a lightemitting part of the display device (for example noise caused by abacklight configuration of a liquid crystal display device). Effects ofthe noise make it difficult to detect a contact position or a proximityposition.

The present invention has been made in view of such points. It isdesirable to easily eliminate effects of extraneous light and noise whenemitting light and receiving light simultaneously.

According to a first embodiment of the present invention, there isprovided a display device for displaying an image and receiving lightsimultaneously or alternately, the display device including: a pluralityof display elements for displaying the image on a display surface of thedisplay device by emitting light; a plurality of light receivingelements for receiving light incident on the display surface formed bythe display elements; a storing unit for storing, as an initial value, adifference between two amounts of light received by the light receivingelements when the display elements emit light and when the displayelements do not emit light in a state of nothing being in contact withor in proximity to the display surface; and a detecting unit fordetecting a state of contact with or proximity to the display surface bydetecting an amount of light received by the light receiving elementswhen the display elements emit light and detecting an amount of lightreceived by the light receiving elements when the display elements donot emit light, and subtracting the initial value stored by the storingunit from a difference between the two detected amounts of receivedlight.

According to a second embodiment of the present invention, there isprovided a display method for displaying an image on a display surfaceand receiving light simultaneously or alternately, the display methodincluding the steps of: displaying the image on the display surface on abasis of light emission for display; performing light reception forreceiving light incident on the display surface, and as the lightreception, performing two light receptions in a state in which the lightemission for display in the step of displaying is performed and a statein which the light emission for display is not performed; storing, as aninitial value, a difference between amounts of light in the two lightreceptions in a state of nothing being in contact with or in proximityto the display surface; and detecting a difference value between amountsof light in the two light receptions to detect a state of contact withor proximity to the display surface, and detecting a state of contactwith or proximity to the display surface by subtracting the initialvalue stored in the step of storing from the detected difference value.

Thus, for example, an amount of received light detected by an element ata position of contact with the display surface in a state of elementsemitting light and an image being displayed on the display surface is asubstantially constant amount of received light irrespective of presenceor absence of extraneous light even when the extraneous light is presentbecause an object itself in contact with the display surface blocks theextraneous light. Amounts of received light detected by elements atother positions are changed from the constant amount of received light.

By subtracting the difference between the amount of received light inthe light emitting state and the amount of received light in thenon-light-emitting state which difference is stored as the initial valuefrom the difference between the two amounts of received light, it ispossible to eliminate effects of internal reflection in the displaydevice, and determine contact with a corresponding position. Similardetermination can be made for a state of proximity to the displaysurface.

According to the present invention, a state of contact with (orproximity to) a corresponding position can be determined on the basis ofthe difference between the two amounts of received light, and effects ofinternal reflection in the display device are eliminated by subtractingthe stored initial value. It is therefore possible to make excellentcontact or proximity determination unaffected by a use environment andthe internal configuration of the device, with effects of extraneouslight and effects inside the display device being eliminated.

The above and other features and advantages of the present inventionwill become apparent from the following description when taken inconjunction with the accompanying drawings which illustrate preferredembodiments of the present invention by way of example.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of configuration of adisplay device according to a first embodiment of the present invention;

FIG. 2 is a block diagram showing an example of a display panelaccording to the first embodiment of the present invention;

FIG. 3 is a connection diagram showing an example of a pixelconfiguration according to the first embodiment of the presentinvention;

FIG. 4 is a connection diagram showing an example of a configuration forreading from pixels according to the first embodiment of the presentinvention;

FIG. 5 is a timing chart showing an example of timing of turning on andoff a backlight according to the first embodiment of the presentinvention;

FIG. 6 is a flowchart of principles of a process for determining acontact or proximity position according to the first embodiment of thepresent invention;

FIG. 7 is a flowchart of an example of a process for determining acontact or proximity position according to the first embodiment of thepresent invention;

FIG. 8 is a diagram of assistance in explaining an example of receivedlight data at a time of self-emitted light being on according to thefirst embodiment of the present invention;

FIG. 9 is a diagram of assistance in explaining an example of receivedlight data at a time of the self-emitted light being off according tothe first embodiment of the present invention;

FIGS. 10A and 10B are diagrams of assistance in explaining an example ofreceived light data at a time of the self-emitted light being on and anexample of received light data at a time of the self-emitted light beingoff without extraneous light according to the first embodiment of thepresent invention;

FIG. 11 is a diagram of assistance in explaining an example of receivedlight data after noise removal according to the first embodiment of thepresent invention;

FIG. 12 is a block diagram showing an example of configuration of adisplay device according to a second embodiment of the presentinvention;

FIG. 13 is a connection diagram showing an example of a pixelconfiguration according to the second embodiment of the presentinvention; and

FIGS. 14A, 14B, 14C, 14D, and 14E are explanatory diagrams showing anexample of drive timing according to the second embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A first embodiment of the present invention will hereinafter bedescribed with reference to FIGS. 1 to 11.

In this example, the present invention is applied to a display deviceformed as a liquid crystal display. A light receiving element isdisposed adjacent to each light emitting element forming the liquidcrystal display so that light emission (display) and light reception(reading) can be performed in parallel with each other. The display inthis example that can perform the light emission and the light receptionin parallel with each other will be referred to as an I/O displaybecause the display can perform image input (light reception) and imageoutput (display) simultaneously. In addition, as will be describedlater, the I/O display in this example can also detect not only anobject in contact with a screen but also an object adjacent to thescreen. Therefore contact detection in the following descriptionincludes proximity detection unless otherwise specified.

FIG. 1 is a block diagram showing an example of configuration of thedisplay device in the present example. An application program executingunit 11 performs a process for displaying an image corresponding to anapplication being executed by the application program executing unit 11.In addition, the application program executing unit 11 detects a contactwith a display panel, and performs a process corresponding to acontacted display part, for example. An instruction to display an imageis supplied from the application program executing unit 11 to a displaydrive circuit 12 to perform driving for displaying the image on the I/Odisplay panel 20.

The I/O display panel 20 is formed as a liquid crystal display. The I/Odisplay panel 20 is a display having transparent electrodes on atransparent substrate such as a glass substrate or the like, and aplurality of pixels (display elements) formed in the form of a matrix ina display area (sensor area) 21 (see FIG. 2). A backlight 15 is disposedon a back surface of the I/O display panel 20. The backlight 15 in thepresent example uses an array of a plurality of light emitting diodes,for example, and the on/off control of the backlight can be performed ata relatively high speed. The on/off control of the backlight isperformed in such a manner as to be interlocked with display driving bythe display drive circuit 12.

The I/O display panel 20 has a plurality of light receiving elementsarranged separately from the display elements. Specifically, forexample, light receiving elements are arranged in the form of a matrixin such a manner as to adjoin the respective display elements in thedisplay area (sensor area) 21. A signal charge accumulated incorrespondence with an amount of light received by the light receivingelements is read by driving from a light reception drive circuit 13. Thelight reception drive circuit 13 has a first frame memory 13 a and asecond frame memory 13 b therewithin for use in a determination processnecessary at a time of reading a light reception signal as laterdescribed. An initial value for determination is stored in the memory 13b of the two frame memories 13 a and 13 b in advance at a time ofmanufacturing the display device, for example.

The light reception signal (differential image signal to be describedlater) read and determined by the light reception drive circuit 13 issent to an image processing unit 14 to determine a contact state or thelike as an image and, as occasion arises, determine a coordinateposition of a contact center or the like. A result of the determination(coordinate data, a recognition result and the like) is sent to theapplication program executing unit 11. The application program executingunit 11 performs a process according to an application being executed.For example, the application program executing unit 11 performs aprocess of displaying a point, an area or the like where contact isdetected in a displayed image.

An example of arrangement of drivers in the I/O display panel 20 in thepresent example will next be described with reference to FIG. 2. The I/Odisplay panel 20 having the transparent display area (sensor area) 21 ata center thereof has a display horizontal driver 22, a display verticaldriver 23, a sensor horizontal driver 24, and a sensor vertical driver25 arranged at four sides of edges of the display area 21, as shown inFIG. 2. The display horizontal driver 22 and the display vertical driver23 are supplied with a display signal and a control clock as data fordisplay. The display horizontal driver 22 and the display verticaldriver 23 drive the display elements arranged in the form of a matrix inthe display area 21. The sensor horizontal driver 24 and the sensorvertical driver 25 are supplied with a clock for reading. The sensorhorizontal driver 24 and the sensor vertical driver 25 supply a lightreception signal read in synchronism with the clock to the lightreception drive circuit 13 via a light reception signal line.

FIG. 3 is a diagram showing a constitution of one of pixels arranged inthe display area 21. The constitution for display of one pixel 31 inthis case has a gate electrode 31 h disposed in a horizontal direction,a drain electrode 31 i disposed in a vertical direction, and a switchingelement 31 a disposed at an intersection of the two electrodes, theswitching element 31 a being connected to a pixel electrode 31 b. Theon/off control of the switching element 31 a is performed by a signalobtained through the gate electrode 31 h. A display state at the pixelelectrode 31 b is set by a signal supplied through the drain electrode31 i.

A light receiving sensor (light receiving element) 31 c is disposed inthe vicinity of the pixel electrode 31 b. The light receiving sensor(light receiving element) 31 c is supplied with a power supply voltageVDD. The light receiving sensor (light receiving element) 31 c isconnected with a reset switch 31 d and a capacitor 31 e. The lightreceiving sensor (light receiving element) 31 c accumulates a chargecorresponding to an amount of received light by the capacitor 31 e afterbeing reset by the reset switch 31 d. A voltage proportional to theaccumulated charge is supplied to a signal outputting electrode 31 j viaa buffer amplifier 31 f in timing in which a reading switch 31 g isturned on, and then output to an outside. The turning on/off of thereset switch 31 d is controlled by a signal obtained by a resetelectrode 31 k. The turning on/off of the reading switch 31 g iscontrolled by a signal obtained by a reading control electrode 31 m.

FIG. 4 is a diagram showing a constitution in which signals read fromlight receiving sensors are supplied to the sensor driver 25. FIG. 4shows three pixels 31, 32, and 33 for red (R), green (G), and blue (B)which pixels are arranged in proximity to each other. Chargesaccumulated by capacitors connected to the light receiving sensors 31 c,32 c, and 33 c of the respective pixels are amplified by respectivebuffer amplifiers 31 f, 32 f, and 33 f, and then supplied to the driver25 via signal outputting electrodes in timing in which reading switches31 g, 32 g, and 33 g are turned on. Constant-current sources 41 a, 41 b,and 41 c are connected to the respective signal outputting electrodes,so that the driver 25 can detect signals corresponding to amounts ofreceived light with a high sensitivity.

FIG. 5 is a diagram showing display and light reception of the I/Odisplay panel 20 in the present example. As shown in FIG. 5, supposingthat an image (a moving image or a still image) is displayed in a frameperiod of 1/60 seconds for one frame, for example, each frame period ishalved, and in the first half period (a period of 1/120 seconds), thebacklight 15 is turned on and a display signal is written to eachdisplay element to display an image for that frame period. In the secondhalf period of each frame period, the backlight 15 is turned off, anddisplay is not made.

As for the reading of a signal resulting from light reception by a lightreceiving element, in each frame period, one reading process isperformed in the first half period in which the backlight is turned on,and one reading process is also performed in the second half period inwhich the backlight is turned off. However, while image signal displayneeds to be made for each frame in a frame period, the reading of lightreception signals does not necessarily need to be performed in one frameperiod and may be performed in a somewhat longer period.

A process performed after light reception signals are thus read willnext be described with reference to a flowchart of FIG. 6. As alreadydescribed with reference to FIG. 5, in the present example, two lightreception signals are read in one frame period. Specifically, a signalis read in a state of self-emitted light (backlight light) being on, anda signal is also read in a state of the self-emitted light (backlightlight) being off. A difference between the two read signals is obtained.As data of this difference, data of a difference measured in a state inwhich there is no extraneous light and there is no surface reflector(that is, there is nothing in contact with or adjacent to the surface)is stored and retained as an initial value in the frame memory 13 b(FIG. 1) in advance.

In the flowchart of FIG. 6, a process from step S21 to step S28 on aright side of the flowchart of FIG. 6 is a process for storing theinitial value in the frame memory 13 b, and a process from step S11 tostep S18 on a left side of the flowchart of FIG. 6 is a process fordetecting contact or proximity using the initial value stored in theframe memory 13 b and a measured value at a time in question. Theprocess for storing the initial value in the frame memory 13 b isperformed before shipment after the display device is manufactured in afactory, for example. Alternatively, the initial value may be updated atany time by a user operation or the like.

Description will first be made of the process for storing an initialvalue in the frame memory 13 b. Data is read in a state of self-emittedlight (backlight light) being on with nothing in contact with oradjacent to the display surface in an environment in which no extraneouslight enters the display surface from the surroundings of the displaydevice (step S21). The data read at this time is stored in the framememory 13 a (step S22). The stored data is retained in the frame memory13 a (step S23). Then, data is also read in a state of the self-emittedlight (backlight light) being off (step S24). The data stored in theframe memory 13 a is subtracted from the data read at this time (stepS25). A difference between the data in the state of the self-emittedlight being on and the data in the state of the self-emitted light beingoff is thereby obtained (step S26). Thus obtained difference data forone screen is stored in the frame memory 13 b (step S27). The storedvalue is retained as an initial value for measurement until the storedvalue is updated (step S28). Incidentally, the data retained by theframe memory 13 a is used in the process of the calculation, and maytherefore be erased after the process is ended.

Then, when measurement is performed to actually detect contact orproximity, data is read in a state of the self-emitted light (backlightlight) being on (step S11). The data read at this time is stored in theframe memory 13 a (step S12). The stored data is retained in the framememory 13 a (step S13). Then, data is also read in a state of theself-emitted light (backlight light) being off (step S14). The datastored in the frame memory 13 a is subtracted from the data read at thistime (step S15). A difference between the data in the state of theself-emitted light being on and the data in the state of theself-emitted light being off is thereby obtained (step S16).

After difference data is obtained in step S16, the initial value formeasurement stored in the frame memory 13 b in step S28 is subtractedfrom the difference data (step S17). Data resulting from the subtractionis obtained as difference data after noise removal (step S18).

Detected difference data is sent as a difference image in a unit of oneframe to the image processing unit 14. The image processing unit 14binarizes the supplied difference image into a value equal to or higherthan a predetermined level and a value lower than the predeterminedlevel in a pixel unit. Further, the image processing unit 14 performs anarithmetic process for determining the center of gravity of an areabinarized and detected as an area having values equal to or higher thanthe predetermined level. The image processing unit 14 sends thecoordinate position of the determined center of gravity to theapplication program executing unit 11. The application program executingunit 11 identifies the supplied coordinate position of the center ofgravity as central position of a contacting or adjacent object.

In the process represented in FIG. 6, the noise measurement process andthe actual contact detection process are represented as differentprocesses. However, since the two processes are actually similar to eachother, the two processes can share a common process as shown in aflowchart of FIG. 7.

Specifically, data is read in a state of self-emitted light (backlightlight) being on (step S11). The data read at this time is stored in theframe memory 13 a (step S12). The stored data is retained in the framememory 13 a (step S13). Then, data is also read in a state of theself-emitted light (backlight light) being off (step S14). The datastored in the frame memory 13 a is subtracted from the data read at thistime (step S15). A difference between the data in the state of theself-emitted light being on and the data in the state of theself-emitted light being off is thereby obtained (step S16).

After difference data is obtained in step S16, whether the process up tostep S16 is a process for obtaining an initial value for noisemeasurement or a process for actually detecting contact or proximity isdetermined (step S31). When the process up to step S16 is the processfor obtaining an initial value for noise measurement, the differencedata is stored in the frame memory 13 b (step S32). The frame memory 13b retains the value as an initial value corresponding to noise (stepS33).

When it is determined in step S31 that the process up to step S16 is aprocess for actually detecting contact or proximity, a value stored inthe frame memory 13 b is subtracted from measurement data at that time(step S17). A resulting subtraction value is handled as difference dataafter noise removal (step S18).

By thus subtracting the noise (initial value) measured in advance fromthe measured value, it is possible to obtain a measured value free fromthe noise dependent on the constitution of the display device(especially the constitution of the backlight), and thus detect acontact position or a proximity position well with high accuracy.

The fact that noise can be removed favorably by the process in thepresent example will be described with reference to examples ofmeasurement data in FIGS. 8 to 11. FIG. 8 is a diagram showing anexample of data for one horizontal line when received light data is readin a state of self-emitted light being on in the display device in thepresent example. In the example of FIG. 8, a signal component d1resulting from a reflection from an object (reflector) in contact withthe display surface is a signal desired to be detected. In actuality,however, a signal component d2 resulting from a backlight internalreflection, a signal component d3 resulting from pixel variations, and asignal component d4 resulting from extraneous light noise are added. Thesignal component d2 resulting from the backlight internal reflectionincludes variations in sensitivity. The signal component d3 resultingfrom pixel variations is a so-called DC noise as base noise added whendetection is performed in any state.

While the signal component d4 resulting from the extraneous light noisedoes not have the signal component d1 to be detected superimposedthereon, the signal component d2 resulting from the backlight internalreflection and the signal component d3 resulting from the pixelvariations have the signal component d1 to be detected superimposedthereon. The signal component d1 to be detected is extracted byperforming a process represented in a flowchart of FIG. 6 or FIG. 7 asdescribed below.

FIG. 9 shows an example of received light data at a time of theself-emitted light being off (that is, at a time of the backlight beingoff). As shown in this example, the signal component d3 resulting fromthe pixel variations and the extraneous light noise d4 are detected atthe time of the self-emitted light being off. At least the signalcomponents d3 and d4 can be removed by subtracting the received lightdata at the time of the self-emitted light being off which data is shownin FIG. 9 from the received light data at the time of the self-emittedlight being on which data is shown in FIG. 8. The subtraction processusing the first frame memory 13 a corresponds to this.

In the present example, a measurement initial value is stored in advancein the second frame memory 13 b, and the initial value is subtracted.This initial value is generated from received light data shown in FIG.10A and received light data shown in FIG. 10B. The received light datashown in FIG. 10A is obtained with the backlight on in a state ofnothing being in contact with the display surface (there being noreflector in contact with the display surface) in an environment freefrom extraneous light, and the received light data shown in FIG. 10B isfurther obtained with the backlight off in this state.

As the received light data at the time of the backlight being on whichdata is shown in FIG. 10A, the signal component d2 resulting from thebacklight internal reflection and the base noise d3 are detected. As thereceived light data at the time of the backlight being off which data isshown in FIG. 10B, the base noise d3 is detected. In the presentexample, a difference between the detection data shown in FIG. 10A andthe detection data shown in FIG. 10B is stored in the second framememory 13 b. Hence, the initial value stored in the second frame memory13 b is a value corresponding to the signal component d2 resulting fromthe backlight internal reflection.

Thus, by subtracting the initial value stored in the second frame memory13 b (that is, the signal component d2 resulting from the backlightinternal reflection) from the difference between the detection data ofFIG. 8 and the detection data of FIG. 9, it is possible to ultimatelyextract the signal component d1 to be detected as shown in FIG. 11.Since the signal component d1 corresponding to contact or proximity canbe thus extracted, contact with or proximity to the display surface ofthe display device can be detected favorably with high accuracy usingthe extracted signal.

Incidentally, while the stored data in the second frame memory 13 bwhich data is used for such noise removal is stored at a time ofmanufacturing the display device in the above-described process, thestored data may be updated at any time during actual use of the displaydevice, for example. Specifically, the stored data may be updated byperforming a process of remeasuring the data stored in the frame memory13 b when there is a certain user operation. While it is ideal if thedata (initial value) first stored in the second frame memory 13 b at thetime of manufacturing the display device or the like is an individualvalue obtained by performing measurement for each display device, arepresentative value may be prepared in advance for each model, and theprepared value may be stored as an initial value for the same model.

In the above-described example, the data stored in the second framememory 13 b is obtained from the difference between the received lightsignal in the case of the self-emitted light being on which signal isobtained by measurement with no extraneous light and the received lightsignal in the case of the self-emitted light being off which signal isobtained by measurement with no extraneous light. However, in principle,measurement may be performed with extraneous light. That is, when thebrightness of the extraneous light (illuminating light) is substantiallyuniform, a similar initial value can be obtained because an initialvalue free from the component resulting from the extraneous light can beobtained by determining the difference between the measurement signal inthe case of the self-emitted light being on and the measurement signalin the case of the self-emitted light being off. However, it isdesirable that the initial value be obtained from a value measuredwithout extraneous light if possible because the extraneous lightcomponent can be varied between the two measurements.

A second embodiment of the present invention will next be described withreference to FIGS. 12 to 14E. While the foregoing first embodiment is anexample in which the present invention is applied to a liquid crystaldisplay, the present embodiment is an example in which the presentinvention is applied to an organic EL display. A liquid crystal displayneeds a backlight as light emitting means separately from displaypixels. In the case of an organic EL display, however, elements formingpixels emit light, and therefore a part of a processing constitutionneeds to be changed from that of the first embodiment. In the presentembodiment, description will focus on this point. The processing of adetected received light signal is the same as in the first embodiment,and therefore description thereof will be omitted.

FIG. 12 is a block diagram showing an example of configuration of adisplay device according to the present embodiment. An applicationprogram executing unit 51 performs a process for displaying an imagecorresponding to an application being executed by the applicationprogram executing unit 51. In addition, the application programexecuting unit 51 detects a contact with a display panel, and performs aprocess corresponding to a contacted display part, for example. Aninstruction to display an image is supplied from the application programexecuting unit 51 to a display drive circuit 52 to perform driving fordisplaying the image on an I/O display panel 60.

The I/O display panel 60 is formed as an organic EL display. The I/Odisplay panel 60 is a display having a plurality of pixels (displayelements) formed in the form of a matrix in a display area (sensorarea). The display elements also function as light receiving elements,and a light emitting period and a light receiving period are set by timedivision. A signal charge accumulated in correspondence with an amountof light received in the light receiving period is read by driving froma light reception drive circuit 53. The light reception drive circuit 53has a first frame memory 53 a and a second frame memory 53 b therewithinfor use in a determination process (difference detecting process)necessary at a time of reading a light reception signal. The secondframe memory 53 b is a memory storing a measurement initial value asdescribed in the first embodiment at a time of shipment from a factory,for example.

The light reception signal (differential image signal) read anddetermined by the light reception drive circuit 53 is sent to an imageprocessing unit 54 to determine a contact state or the like as an imageand, as occasion arises, determine a coordinate position of a contactcenter or the like. A result of the determination (coordinate data, arecognition result and the like) is sent to the application programexecuting unit 51. The application program executing unit 51 performs aprocess according to an application being executed. For example, theapplication program executing unit 11 performs a process of displaying apoint, an area or the like where contact is detected in a displayedimage.

FIG. 13 is a diagram showing a configuration of one pixel. In FIG. 13, alight emitting element 61 in the organic EL display is shown as a lightemitting diode. A parasitic capacitance 61 a occurs at the lightemitting element 61. For image display, display data is supplied from adisplay data signal line 62 to the light emitting element 61 via aswitch SW1. Hence, a display period (light emitting period) is set by aperiod during which the switch SW1 is on.

During a period which the light emission of the light emitting element61 is stopped, a charge is accumulated in the parasitic capacitance 61 aoccurring at the light emitting element 61 according to an amount oflight incident on a surface of the display panel. The accumulated chargeis output to a reception data signal line 64 when a switch SW2 is turnedon. Incidentally, at a time of a start of a light receiving period, aswitch SW3 for reset needs to be turned on for a moment to discharge acharge accumulated in the parasitic capacitance 61 a during lightemission. The turning on of the switch SW2 is controlled by a signalobtained in a reading line selecting line 63.

FIGS. 14A, 14B, 14C, 14D, and 14E show an example of detecting contactwith or proximity to such an organic EL display type I/O display panel60 while displaying an image or the like on the I/O display panel 60.

FIG. 14A shows a state of the I/O display panel 60. In this figure, thesurface of the I/O display panel 60 is touched with a finger f. In thestate shown in FIG. 14A, a light emitting area 60 a is a plurality ofspecific horizontal lines within one screen. While light is not emittedin the light emitting area 60 a, the light emitting area 60 a is changedwithin one field period, so that a person viewing the screen can see thedisplay of the entire screen due to an afterimage effect. FIG. 14A showsthe horizontal lines of the light emitting area 60 a being changed froma top to a bottom.

In this state, as for reading of a received light signal, two readings,that is, a reading of a horizontal line 60 b on an upper side adjacentto the light emitting area 60 a and a reading of a horizontal line 60 cat a certain distance from the horizontal line 60 b are performed in onefield period. The read lines 60 b and 60 c are also changed in order insuch a manner as to be interlocked with the changing of the lightemitting area 60 a.

By performing such reading, the reading of the horizontal line 60 badjacent to the light emitting area 60 a is a reading that can detectreflection of light from the light emitting area 60 a, thus providingread data at a time of self-emitted light being on, as shown in FIG.14B. The reading of the horizontal line 60 c distant from the lightemitting area 60 a is a reading free from effects of light emission,thus providing read data at a time of the self-emitted light being off,as shown in FIG. 14C. Hence, a difference between the two pieces of readdata is detected as shown in FIG. 14D, and thereby a difference betweenthe received light data at the time of the self-emitted light being onand the received light data at the time of the self-emitted light beingoff is detected. The data of an initial value measured in advancewithout extraneous light and stored in the frame memory 53 b issubtracted from image data on that difference, whereby noise-removeddifference data is obtained as shown in FIG. 14E. As a concrete exampleof the process of obtaining the noise-removed difference data from thereceived light data, the process represented in the flowchart of FIG. 6or FIG. 7 described in the foregoing first embodiment can be applied.Thus, with the constitution of the present example, as in the foregoingfirst embodiment, it is possible to remove effects of extraneous light,eliminate degradation in sensitivity due to nonuniformity ofcharacteristics of elements forming pixels, and remove noise of theconstitution of a light source itself. Therefore contact or proximitycan be detected excellently.

Thus, the present invention is applicable to a case where displayelements forming pixels are light emitting elements as in an organic ELdisplay. The present invention is therefore applicable to any of caseswhere light emitting means separate from a display panel is necessary asin a liquid crystal display and cases where a display panel itself emitslight. While in the foregoing embodiments, description has been made bytaking a liquid crystal display and an organic EL display as examples ofthe respective display panels, the present invention is applicable todisplays having other constitutions as long as light receiving elementscan be incorporated into the displays.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on designs and other factors insofar as they are within thescope of the appended claims or the equivalents thereof.

1. A display device for displaying an image and receiving lightsimultaneously or alternately, said display device comprising: aplurality of display elements for displaying the image on a displaysurface of said display device by emitting light; a plurality of lightreceiving elements for receiving light incident on the display surfaceformed by said display elements; a storing unit for storing, as aninitial value, a difference between two amounts of light received bysaid light receiving elements when said display elements emit light andwhen said display elements do not emit light in a state of nothing beingin contact with or in proximity to said display surface; and a detectingunit for detecting a state of contact with or proximity to the displaysurface by detecting an amount of light received by said light receivingelements when said display elements emit light and detecting an amountof light received by said light receiving elements when said displayelements do not emit light, and subtracting said initial value stored bysaid storing unit from a difference between the two detected amounts ofreceived light.
 2. The display device as claimed in claim 1, furthercomprising a backlight for illuminating said display surface from aback, said backlight being separate from said display elements, whereintwo light receptions for obtaining said initial value and two lightreceptions for obtaining a detection value are light reception in astate of the backlight illuminating and light reception in a state ofthe backlight not illuminating.
 3. The display device as claimed inclaim 1, wherein a value obtained by subtracting said initial valuestored by said storing unit is binarized, and a position of contact withor proximity to the display surface is detected.
 4. The display deviceas claimed in claim 1, wherein the initial value stored by said storingunit is updated on a basis of a predetermined operation.
 5. The displaydevice as claimed in claim 1, wherein said display elements and saidlight receiving elements are identical elements; said display elementsare used as light receiving elements in a period when light emissiondisplay by said display elements is not made; and light reception whenlight emission by said display elements is performed is light receptionimmediately after the light emission by said display elements isstopped, and light reception when said light emission is not performedis light reception after passage of a predetermined time after the lightemission is stopped.
 6. A display method for displaying an image on adisplay surface and receiving light simultaneously or alternately, saiddisplay method comprising the steps of: displaying the image on saiddisplay surface on a basis of light emission for display; performinglight reception for receiving light incident on said display surface,and as the light reception, performing two light receptions in a statein which said light emission for display in said step of displaying isperformed and a state in which said light emission for display is notperformed; storing, as an initial value, a difference between amounts oflight in said two light receptions in a state of nothing being incontact with or in proximity to the display surface; and detecting adifference value between amounts of light in said two light receptionsto detect a state of contact with or proximity to the display surface,and detecting a state of contact with or proximity to the displaysurface by subtracting the initial value stored in said step of storingfrom the detected difference value.